{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "00b38838-5730-4cb8-860a-c1af45bb685a",
   "metadata": {},
   "source": [
    "# L5: Industry Applications"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c32e7923",
   "metadata": {},
   "source": [
    "<p style=\"background-color:#fff6e4; padding:15px; border-width:3px; border-color:#f5ecda; border-style:solid; border-radius:6px\"> ⏳ <b>Note <code>(Kernel Starting)</code>:</b> This notebook takes about 30 seconds to be ready to use. You may start and watch the video while you wait.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76dac9eb",
   "metadata": {},
   "source": [
    "* In this classroom, the libraries have been already installed for you.\n",
    "* If you would like to run this code on your own machine, you need to install the following:\n",
    "```\n",
    "    !pip install google-generativeai\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "336183fc-6bb3-4121-b64f-33214c9ef376",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67d16026-4eb6-4282-ae8d-4fb7871959c1",
   "metadata": {
    "height": 252
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "GOOGLE_API_KEY=os.getenv(\"GOOGLE_API_KEY\")\n",
    "\n",
    "import google.generativeai as genai\n",
    "from google.api_core.client_options import ClientOptions\n",
    "genai.configure(\n",
    "    api_key=GOOGLE_API_KEY,\n",
    "    transport=\"rest\",\n",
    "    client_options=ClientOptions(\n",
    "        api_endpoint=os.getenv(\"GOOGLE_API_BASE\"),\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f01460f",
   "metadata": {},
   "source": [
    "> Note: learn more about [GOOGLE_API_KEY](https://ai.google.dev/) to run it locally."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46f256f9-ecd9-45e7-b8cb-8d3815df8f28",
   "metadata": {},
   "source": [
    "## Vision Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e50df417-b39b-48e9-af89-c39ff4269f16",
   "metadata": {
    "height": 336
   },
   "outputs": [],
   "source": [
    "import textwrap\n",
    "import PIL.Image\n",
    "from IPython.display import Markdown, Image\n",
    "\n",
    "def to_markdown(text):\n",
    "    text = text.replace(\"•\", \"  *\")\n",
    "    return Markdown(textwrap.indent(text, \"> \", predicate=lambda _: True))\n",
    "\n",
    "def call_LMM(image_path: str, prompt: str, plain_text: bool=False) -> str:\n",
    "    img = PIL.Image.open(image_path)\n",
    "\n",
    "    model = genai.GenerativeModel(\"gemini-pro-vision\")\n",
    "    response = model.generate_content([prompt, img], stream=True)\n",
    "    response.resolve()\n",
    "    \n",
    "    if(plain_text):\n",
    "        return response.text\n",
    "    else:\n",
    "        return to_markdown(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a3ffe26-0b19-432e-9c18-7ae6ac41b437",
   "metadata": {},
   "source": [
    "## Extracting Structured Data from Retreived Images\n",
    "### Analyzing an invoice"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "561ed800-0315-42bc-aee5-9b1ed7bb0926",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "Image(url=\"invoice.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "533541f1-79a2-48b7-b562-ac6eb2a76e88",
   "metadata": {},
   "source": [
    "<p style=\"background-color:#fff6ff; padding:15px; border-width:3px; border-color:#efe6ef; border-style:solid; border-radius:6px\"> 💻 &nbsp; <b>Access Files and Helper Functions:</b> To access the files for this notebook, 1) click on the <em>\"File\"</em> option on the top menu of the notebook and then 2) click on <em>\"Open\"</em>. For more help, please see the <em>\"Appendix - Tips and Help\"</em> Lesson.</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63155f19-9e0f-4add-beb9-c773486d19dd",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "call_LMM(\"invoice.png\",\n",
    "    \"\"\"Identify items on the invoice, Make sure you output \n",
    "    JSON with quantity, description, unit price and ammount.\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f332676-9a5a-4ef7-b27e-cfc50018e589",
   "metadata": {
    "height": 115
   },
   "outputs": [],
   "source": [
    "# Ask something else\n",
    "call_LMM(\"invoice.png\",\n",
    "    \"\"\"How much would four sets pedal arms cost\n",
    "    and 6 hours of labour?\"\"\",\n",
    "    plain_text=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d7e2215-be23-4d11-8ef9-26e88720f66c",
   "metadata": {},
   "source": [
    "### Extracting Tables from Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee468b2d-735e-46a9-9de5-e0f158d4412f",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "Image(\"prosus_table.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c278becb-c206-4be5-83d1-8f58f0a5435e",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "call_LMM(\"prosus_table.png\", \n",
    "    \"Print the contents of the image as a markdown table.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04bbe338-5bb6-4814-bd13-d19ad093fdf8",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "call_LMM(\"prosus_table.png\", \n",
    "    \"\"\"Analyse the contents of the image as a markdown table.\n",
    "    Which of the business units has the highest revenue growth?\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c40eef4-3898-4751-951e-48633fdde1cd",
   "metadata": {},
   "source": [
    "### Analyzing Flow Charts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85f52b14-e783-4375-a136-52d556fd1f92",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "Image(\"swimlane-diagram-01.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ceb04606-1d5f-41b3-bde7-a397ef052a2e",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "call_LMM(\"swimlane-diagram-01.png\", \n",
    "    \"\"\"Provide a summarized breakdown of the flow chart in the image\n",
    "    in a format of a numbered list.\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8be1f359-a3eb-4584-8cf2-4b13e29d51e2",
   "metadata": {
    "height": 81
   },
   "outputs": [],
   "source": [
    "call_LMM(\"swimlane-diagram-01.png\", \n",
    "    \"\"\"Analyse the flow chart in the image,\n",
    "    then output Python code\n",
    "    that implements this logical flow in one function\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9821365e",
   "metadata": {},
   "source": [
    "* Test the code generate above.\n",
    "> Note: please be advised that the output may include errors or the functionality may not be fully operational, as it requires additional inputs to function properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a84268f8-117a-44e9-b281-29e96d06d102",
   "metadata": {
    "height": 421
   },
   "outputs": [],
   "source": [
    "def order_fulfillment(client, online_shop, courier_company):\n",
    "   # This function takes three objects as input:\n",
    "   # - client: the client who placed the order\n",
    "   # - online_shop: the online shop that received the order\n",
    "   # - courier_company: the courier company that will deliver the order\n",
    "\n",
    "   # First, the client places an order.\n",
    "   order = client.place_order()\n",
    "\n",
    "   # Then, the client makes a payment for the order.\n",
    "   payment = client.make_payment(order)\n",
    "\n",
    "   # If the payment is successful, the order is shipped.\n",
    "   if payment.status == \"successful\":\n",
    "       online_shop.ship_order(order)\n",
    "       courier_company.transport_order(order)\n",
    "   \n",
    "   # If the payment is not successful, the order is canceled.\n",
    "   else:\n",
    "       online_shop.cancel_order(order)\n",
    "       client.refund_order(order)\n",
    "\n",
    "   # Finally, the order is invoiced.\n",
    "   online_shop.invoice_order(order)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
